Adversarial Training Reduces Information and Improves Transferability

Recent results show that features of adversarially trained networks for classification, in addition to being robust, enable desirable properties such as invertibility. The latter property may seem counter-intuitive it is widely accepted by the community classification models should only capture minimal information (features) required task. Motivated this discrepancy, we investigate dual relationship between Adversarial Training and Information Theory. We can improve linear transferability new tasks, from which arises a trade-off representations accuracy on source validate our employing robust CIFAR-10, CIFAR-100 ImageNet several datasets. Moreover, reduces Fisher about input weights task, provide theoretical argument explains invertibility deterministic without violating principle minimality. Finally, leverage insights remarkably quality reconstructed images through inversion.